User interface (UI) is often one of the most important parts of a computer program because it determines how easily a user can communicate with the program. A powerful program with a poorly designed UI has little value. Text-based and graphical user interfaces (GUIs) that use windows, icons, and pop-up menus have become standard on personal computers. Text-based UIs as well as GUIs typically use an input device, such as a keyboard, mouse or stylus, to provide user input and control the movement of a cursor or pointer on a display screen.
Touch-sensitive surfaces are rapidly becoming more common in computing devices. A natural input device for computing devices with touch-sensitive surfaces is a user's finger. They are very convenient as they allow a user to make natural gestures familiar to the user in other contexts, such as by entering handwriting using a special stylus such as an elongated pen-like object having a pointed tip. Many of these devices also allow input to be provided by user's fingertip. The term touch-sensitive surface or device will be used herein to refer to such surfaces or devices that are configured to detect the touch of any type of “stylus” such as a specially-design stylus device or a user's finger. One disadvantage of using a finger or other broad-tipped stylus with a touch-sensitive device is that it is more difficult to accurately target where the finger will and will not touch. For instance, where the touch-sensitive device is combined with a display (thus, a touch-sensitive display), it is difficult to use a finger to target small displayed objects so that they may be selected and/or manipulated. Note that for many elderly and for some people with neurological disorder, it is very difficult to stop their hands from shaking. Clearly, constantly shaking hand will make it very hard for a user to use his finger to navigate through information displayed densely on a touch screen.
In addition, various pointing devices such as mice and man-made pen-type stylus digitizers provide for a hover function, meaning that the user may position the cursor over an area without generating a click. However, few if any touch-sensitive displays have this capability when used by a human finger. The need exists to provide a way for users to hover the cursor using only their finger on a touch-sensitive display, and to provide a way for users to be able to use a touch-sensitive device with their fingers or other inherently imprecise styluses, while maintaining a reasonable level of accuracy.
Solutions including a patent assigned to Microsoft Corporation are attempting to address this problem. One of such patents is a recently issued U.S. Pat. No. 7,489,306 “Touch screen accuracy” by Kolmykov-Zotov, et al. In this invention, a user interface is provided allowing a user to simulate mouse hovering with the user's finger on a touch-sensitive display. When a finger or other stylus touches the touch-sensitive display adjacent to a display cursor, a control appears that enables the user to drag the cursor about the display by dragging the control. Manipulation of the control instead of the cursor allows the user to see the cursor while it moves, making it easier to accurately target small objects that were previously difficult to target directly. See FIG. 1 copied from the above mentioned patent application.
FIG. 1 illustrates an invention described in U.S. Pat. No. 7,489,306 “Touch screen accuracy” by Kolmykov-Zotov, et al. In this embodiment, the cursor handle 192 is a simple circle surrounding the cursor 190. As shown, the contact area 194 is on the cursor handle 192 such that the cursor 190 is ready to be moved. As the user moves his or her finger downward and to the right in FIG. 1, for example, the cursor handle 192 and the cursor 190 simultaneously move with the finger and in the same direction as the finger. At some point during the movement, the finger has now moved to a location indicated by 194a, the cursor 190 has moved to a position indicated by 190a, and the cursor handle 192 has moved to a position indicated by 192a. During the movement, the cursor 190 remains adjacent to the finger (i.e., adjacent to the contact area 194) and the cursor handle 192.
Unfortunately, solution of U.S. Pat. No. 7,489,306 described above is not well suited for small touch screen devices such as cell phones and PDAs with relatively large displayed information. Information that can be easily navigated on large touch screens, need to be reduced in size if the same level of navigation convenience is expected on smaller touch screens. FIG. 2A shows a scaled down version of a city map on a small screen normally seen well on larger screens.
Same information can be easily viewed on larger screen as illustrated in FIG. 2B. Prior art solutions to such problems are re-formatting of information to fit smaller size touch screens, or to zoom into a specific area as needed. The first type of solution is described, for example, in U.S. Pat. No. 6,742,043 “Reformatting with modular proxy server” by Moussa, et al. or in U.S. Pat. No. 6,226,642 “Content modification of internet web pages for a television class display” by Beranek, et al. Authors of both inventions realize that information normally formatted for certain type of screens, needs to be displayed on screens of different sizes and resolution capabilities. Therefore they propose solutions addressing such needs by properly reformatting information content. Any one skilled in the art would recognize immediately that reformatting data is costly as it requires additional hardware and software. Moreover, there is added cost of maintenance as reformatted data could quickly become outdated. E.g., news information may need to be constantly updated, thus reformatting data would be short lived. Moreover, for small devices such as cell phones, PDAs, etc, reformatting most likely would mean reducing the content. Therefore, richness of the content will be lost, and navigating through limited information displayed on the screen may not be very valuable service for the users.
Another obvious and commonly used solution is to zoom into specific area, of displayed information. Examples are seen in daily life of anyone using computers zooming into the text of online documentation, or zooming into specific area of a map, etc. For example, you can double-tap or use the pinch gesture to expand webpage columns in Safari. Zoom is a special accessibility feature that lets you magnify the entire screen of any application. For example iPhone by APPLE Corporation is used to see what's on the display. Turn Zoom on or off: In Settings, choose General>Accessibility>Zoom and tap the Zoom On/Off switch. Zoom in or out: Double-tap the screen with three fingers. By default, the screen is magnified by 200 percent. If you manually change the magnification (by using the tap- and-drag gesture, described below), iPhone automatically returns to that magnification when you zoom in by double-tapping with three fingers.
Disadvantage of this method is that while user zooms into a specific area, he is not able to see the “big picture”. Consider for example a map application. Often user would like to zoom into a specific area on a city map without expanding the whole map, so that he can keep track of the landmarks and their relative distance and location to the zoomed in area. In general, when detailed information is being looked up, it is desirable to keep the bigger picture in perspective.
U.S. Pat. No. 6,992,661 “Electronic device, digital still camera and display control method” by Ikehata et al, describes an electronic device that comprises a touch screen configured to output coordinate information corresponding to the position where it is touched, a display unit configured to display an item indicating an operating area in a predetermined position on the touch screen, and a control unit configured to clear the item displayed on the display unit when an area of the touch screen other than the area indicated by the item is touched. This invention describes a method of selecting and displaying an enlarged image of a part of the image corresponding to the input image data, does this for images in the context of small sized cameras. Ikehata's invention does not describe consecutive zoom in zoom out operations for localized area (as opposed for the whole information displayed by the application). Method for instructing local zoom in/zoom out operations are different. Ikehata's invention does not describe operations with two or more localized zoom-in areas and objects selected in these areas.
There is therefore a need to provide a way for users to be able to use a touch-sensitive device with their fingers or other inherently imprecise styluses, while maintaining a reasonable level of accuracy.